Skip to content

New -O4 tree-level optimization passes, x86 peepholes, and supporting fixes#13

Open
joaopauloschuler wants to merge 287 commits into
fpc-unleashed:mainfrom
joaopauloschuler:main
Open

New -O4 tree-level optimization passes, x86 peepholes, and supporting fixes#13
joaopauloschuler wants to merge 287 commits into
fpc-unleashed:mainfrom
joaopauloschuler:main

Conversation

@joaopauloschuler

@joaopauloschuler joaopauloschuler commented Jul 9, 2026

Copy link
Copy Markdown

New -O4 tree-level optimization passes, x86 peepholes, and supporting fixes

I have not properly tested. This is for you to know what I am working on.

Summary

This PR adds a family of new optimization passes to the Free Pascal compiler, enabled at -O4 (each individually controllable via its own -Oo… / -OoNO… switch) or opt-in via their own switch, together with x86-specific code-generation improvements, interprocedural analyses with cross-unit PPU summaries, bug fixes uncovered by self-compiling the compiler at -O4, and an extensive regression-test suite under unleashed/tests/ (~320 files changed, ~40,000 insertions across 69 commits).

New tree-level optimizer passes (-O4)

Loop optimizations (compiler/optloop.pas, ~6,000 new lines)

  • -OoLICM — loop-invariant code motion: hoists invariant computations out of loops.
  • -OoLOOPUNSWITCH — loop unswitching: moves loop-invariant conditionals out of the loop, duplicating the loop body per branch (with a size cap).
  • -OoLOOPPEEL — full peeling of small constant-trip-count loops.
  • -OoLOOPSPLIT — loop splitting on an induction-variable crossover point.
  • -OoLOOPFUSE — fusion of two adjacent loops iterating over the same space.
  • -OoLOOPDISTPAT — loop-distribution pattern idiom recognition.
  • -OoUNROLLJAM — outer-loop unroll-and-jam.
  • -OoPREDCOM — predictive commoning of cross-iteration reloads.
  • -OoIFCONVERT — loop if-conversion to packed maxps/minps.
  • -OoREASSOC — floating-point reduction reassociation (gated on fast-math).
  • -OoUNROLLDYN (opt-in) — dynamic-trip loop unrolling: unrolls a counted for-loop of unknown trip count by 4 with a scalar remainder loop, keeping the original serial evaluation order so FP accumulation stays bit-identical.
  • -OoPREFETCH (opt-in) — software prefetch: inserts a PREFETCHNTA of base[i+64] once per iteration group for each distinct streamed dynamic-array base (skipping pure store destinations); semantically a no-op.
  • -OoFINALVALUE (opt-in) — final value replacement + dead loop elimination (gcc -ftree-scev-cprop, new unit compiler/optfinalvalue.pas): a counted for-loop whose body is only loop-invariant accumulator updates of plain locals, with a dead counter, is replaced by the closed form if a<=b then s := s + (b-a+1)*c and the loop deleted. Extended to pointer-stride accumulators (inc(p,stride)), multi-statement bodies (one closed form per distinct accumulator), 64-bit counters (exact mod 2^64 arithmetic), static/global accumulators, and native full-range accumulators under -Cr; disabled under -Co by design (the closed form cannot reproduce the per-iteration overflow trap).

Vectorization

  • -OoVECTORIZE — conservative SSE loop autovectorization of element-wise array loops (add/mul/sub, copy, and scalar-broadcast shapes), with alias, range-check, and special-value safety guards. Includes per-loop diagnostics (notes 06065/06066) explaining why a loop was or was not vectorized.
  • Reduction vectorization — extends -OoVECTORIZE to the two canonical single-precision reductions, s := s + a[i] (sum) and s := s + a[i]*b[i] (dot product), widened to a 4-lane packed SSE/AVX accumulator with a scalar residue loop; recognizes the FMA-contracted dot shape on FMA targets. Fast-math gated (it reassociates FP adds, like -OoREASSOC). New note 06086.
  • Double-precision vectorization — extends -OoVECTORIZE from single-only to double dynamic arrays, 2 lanes per 128-bit SSE/AVX window: every existing shape (array-array + - *, scalar-broadcast, copy, and the sum/dot reductions including the FMA-contracted form) now fires on double, with movupd/addpd/subpd/mulpd and a 2-lane horizontal sum in the backend. Any single/double mix is rejected and stays scalar; all single-precision soundness gates are preserved.
  • -OoSLP (opt-in) — superword-level parallelism (gcc tree-slp-vectorize / LLVM SLPVectorizer): packs a run of ≥4 adjacent, isomorphic scalar single-precision assignments over consecutive elements of the same array base — hand-unrolled straight-line code with no surrounding loop — into one 128-bit SSE packed op, reusing the loop vectorizer's backend node. Bit-identical to scalar (no reassociation, no fast-math gate).

Value-range and idiom recognition

  • -OoRANGEELIM — value-range-analysis-driven range-check elimination (including dynamic-array High/Length-1 bounds and must-check preservation for off-by-one and variable bounds).
  • -OoVRP — interval value-range propagation with branch folding: forward-propagates integer intervals seeded from declared subrange/ordinal type bounds, for-loop counter bounds, and straight-line const/mod/and facts, then folds every if the intervals already decide, deleting the dead arm.
  • -OoBITIDIOM — recognition of bit-manipulation idioms: population count, and tzcnt/bsr bit-scan loops.
  • Signed mod by a non-power-of-two constant: magic-number strength reduction (compiler/x86/nx86mat.pas).
  • Strength reduction of loopvar * invariant multiplies inside array-index / pointer-offset expressions.

Scalars, stores, and control flow

  • -OoSRA — scalar replacement of non-escaping local aggregates (new unit compiler/optsra.pas).
  • -OoSTOREMERGE — coalescing of adjacent narrow stores into wider stores.
  • -OoJUMPTHREAD — jump threading / elimination of nested re-tests of the same condition.
  • Extended dead-store elimination (compiler/optdeadstore.pas) — DSE now also handles record-field and static-array-element stores.
  • -OoSINK — partially-dead code sinking (the symmetric counterpart of LICM): a pure assignment immediately preceding an if whose value is consumed on only one arm (and dead after the branch) is relocated into that arm, so paths that never use it stop paying for it.
  • -OoSTOREMOTION — loop store motion / scalar promotion (gcc -fgcse-sm): a global of unmanaged scalar type repeatedly loaded/stored in a loop is promoted to a register temp — loaded once before the loop, stored once after — under a strict no-call/no-alias/no-trap whitelist over the entire loop.
  • -OoGVNPRE (opt-in) — dominator-based global value numbering / full-redundancy elimination across statements and rejoining branches: a side-effect-free scalar expression available on every path to a point is computed once into a temp and reused. Availability tables are copied into branch arms and intersected at rejoins; killed conservatively by operand writes, memory stores, and calls. Validated by a 20k-case randomized differential and a self-compile with the pass active (816 firings). New note 06085.
  • -OoPARTIALINLINE (opt-in) — partial inlining / function splitting (gcc -fpartial-inlining, new unit compiler/optpartialinline.pas): a routine that begins with a cheap, side-effect-free early-exit guard followed by an expensive body is split into a tiny inlinable header (guard + forwarding call) that takes over the original procsym, and an out-of-line body keeping the full original code — so the common guarded-exit path pays no call at all. Covers both procedures and functions (the function case gives the header its own funcret local and remaps the guard's exit(value) onto it, so both arms write the same result location). A 97%-early-exit guard in a hot loop runs ~16% faster.
  • -OoSTACKALLOC (opt-in) — escape-analysis stack allocation of non-escaping local dynamic arrays: a local dynamic array of unmanaged element type whose only SetLength has a positive constant length (within a 4 KiB frame budget) and which provably never escapes is given a frame-local buffer with FPC's own read-only refcount sentinel (-1), eliding the heap allocation entirely; scope-end finalization neither frees nor finalizes it.

Interprocedural analyses and cross-unit PPU summaries

  • -OoPURE (opt-in) — interprocedural pure/const discovery (gcc -fipa-pure-const, new unit compiler/optpure.pas): walks the unit's routines bottom-up over the call graph and proves per routine whether it is const (result depends only on by-value parameters) or pure (may read but never writes global memory, no I/O or other side effects), with a greatest-fixpoint DFS so mutually-recursive SCCs are still proven. Anything not provably harmless is impure. Eligibility covers standalone routines, const/constref-aggregate readers, and read-only instance/class methods (the classic field getter). Emits -vh purity hints (messages 06087/06088). Consumed by:
    • -OoLICM — hoists proven-const calls with loop-invariant arguments out of loops;
    • -OoGVNPRE — CSEs identical pure calls (including direct pure method calls), value-numbered as memory readers killed by any intervening store or impure call;
    • extended dead-store elimination — pure/const calls are no longer hard barriers: pending stores survive a const call entirely, and survive a pure call when the slot is not globally reachable.
  • -OoIPARA (opt-in) — interprocedural register allocation (gcc -fipa-ra): records each generated routine's proven volatile-register clobber set, and narrows the caller-save spills around a direct call to that set, so caller values in untouched volatile registers survive without a spill/reload. Conservative full-ABI-mask fallbacks for indirect/virtual/external/assembler callees, exception contexts, and not-yet-generated routines.
  • -OoICF (opt-in) — identical code folding (gcc -fipa-icf / gold --icf): canonicalizes each generated routine body at the assembler-list level and replaces byte-identical duplicates with a 5-byte jmp thunk to the kept copy (addresses stay pairwise distinct, so @f<>@g survives). A proven address-never-observed, unit-private duplicate is instead collapsed to a zero-byte symbol alias at the survivor's address.
  • Shared cross-unit PPU optimizer summaries — a generic, self-describing per-procdef (tag,len,payload) summary blob streamed inside the PPU ibprocdef entry (versioned once via CurrentPPULongVersion 30 → 31; new passes add tags without further bumps), dumped by ppudump. Consumed for cross-unit -OoPURE (pure/const CSE of used-unit calls, verdict kept in the interface CRC), cross-unit -OoIPARA (clobber mask guarded by a target/ABI/-Cf signature), and cross-unit -OoICF (128-bit digests of surviving routine bodies; a digest-matching routine in a later unit folds to a jmp thunk to the external symbol).

Case-statement lowering

  • -OoCASECLUSTER — gcc-style case clustering: partitions sorted case labels by dynamic programming into an optimal mix of jump-table clusters, bit-test clusters (the case c of 'a','e','i','o','u' shape becomes one shift+AND membership test), and plain-compare singletons, dispatched via a balanced binary comparison tree.
  • -OoSWITCHTABLE — case-to-lookup-table conversion (gcc -ftree-switch-conversion): a fully-covered, hole-free case whose arms only assign compile-time constants to the same ordered set of ordinal variables is replaced by per-variable static lookup tables plus a single range guard (no guard at all under full type coverage). New notes 06083/06084.

x86 code generation

  • Sibling-call frame reuse (compiler/x86/aoptx86.pas): tail calls from a framed routine now reuse the caller's frame, with negative guards for managed types, pointer arguments, and try/finally contexts.
  • -OoSIBCALL (opt-in) — sibling-call optimization (gcc -foptimize-sibling-calls), distinct from the self-recursion rewrite: when a routine's last action is a direct tail call to another routine, the frame teardown is hoisted above the call and the call rewritten to a jmp, so O(depth) stack becomes O(1) for mutual recursion and continuation-style dispatch (an even/odd pair to depth 10⁷ completes under ulimit -s 512). Handles result forwarding through a chain of callee-saved registers and/or frame slots (including 128-bit rax:rdx record results). Hardened frame-teardown gate: any lea/mov materialising a frame-relative address into a general register (escaping compiler temporaries such as open-array-of-const) declines the transform.
  • -OoCROSSJUMP — cross-jumping / tail merging (gcc -fcrossjumping) as a late post-peephole pass: predecessor blocks ending in an operand-exact identical instruction tail keep one copy and jump into it. Pure code-size/I-cache win; refuses to delete regions containing labels, CFI directives, or debug/unwind data.
  • -OoBLOCKORDER — static hot/cold basic-block layout (gcc -freorder-blocks): sinks cold error/raise regions (guard clauses ending in fpc_raiseexception, RunError, range/overflow helpers, etc.) out of the hot path to the end of the routine, inverting the guard so the hot successor is the fall-through.
  • -OoREE — redundant sign/zero-extension elimination (gcc -free) as a post-peephole pass: deletes a movzx/movsx whose value already carries the required extension from its nearest definition (an earlier extension, an and-mask, xor reg,reg, or a small non-negative immediate move), reaching past the adjacency limit of the existing peepholes.
  • -OoSHRINKWRAP (opt-in) — shrink-wrapping (gcc -fshrink-wrap): sinks a push-only callee-saved prologue below an initial volatile-only guard clause, so early-exit fast paths (nil check, zero-length, cache hit) run prologue-free with a bare ret. DWARF CFI stays correct (position labels relocate with the pushes); validated with objdump --dwarf=frames and a full self-host with the pass forced on.

Managed types

  • -OoREFELIDE (opt-in) — managed refcount elision (ARC-style pair elimination): an ansistring borrow a := b where a is thereafter only read is lowered to a plain pointer copy with the incref and the finalization decref both removed; the source provably keeps the buffer alive for a's lifetime. Verified leak-free with -gh on every fixture.

Bug fixes and infrastructure

  • opttree: fixed normalize() hoisting block-expressions out of control-flow bodies, which broke -Oodeadstore on if-expressions and match inside loops.
  • REASSOC: fixed a miscompile / internalerror 200306031 when the loop counter appears in the accumulated expression — substituted (i+delta) copies kept stale cached resultdef/pass1 flags; the whole substituted expression is now re-firstpassed.
  • nadd.pas const-fold: fixed a spurious "Overflow in arithmetic operation" error at -O2+ when the level-2 range-test rewrite's nf_internal unsigned wraparound constants (a negative low bound stored as qword) fold at an inlined call site — internal nodes now keep the wrapped value instead of raising; user-level overflow reporting is unchanged. (Made neural-api's neuralvolume.pas compilable at -O2+.)
  • optvectorize: fixed a -Oodeadstore × -OoVECTORIZE reduction miscompile — the packed accumulator's lane-0 seed read of s was embedded in a backend node invisible to DFA, so DSE deleted the incoming s := <start> store; the seed now goes through a plain DFA-visible scalar copy.
  • testtool: fixed the run phase on unix targets (.exe was appended unconditionally) and injected -Facthreads on non-Windows so threaded-feature tests link the cthreads driver; 1200/1202 tests now pass on x86_64-linux.
  • optloop.pas: satisfied un_main's stricter case-exhaustiveness check after cherry-picking the optimizer work.
  • Made an -O4 stage-2 self-compile of the compiler build clean: fixed DFA false positives and case-exhaustiveness warnings.
  • Inline variables: infer Double for real-literal initializers; documented the Rtti unit build for tests.
  • New/updated diagnostics in compiler/msg/errore.msg; ppudump updated for the new option flags.
  • Developer tooling: fpcu.sh (invoke the in-tree ppcx64 with the built unit paths) and lazbuildu.sh (lazbuild using the in-tree compiler) wrapper scripts; README Quick Start section on compiling programs and Lazarus projects with -O4.

Testing

  • Each pass ships with dedicated test files under unleashed/tests/testfiles/ (optlicm, optunswitch, optvect, optvrp, optvrpfold, optbitidiom, optbitscan, optsibcall, optstoremerge, optdistpat, optreassoc, optcasecluster, optcrossjump, optblockorder, optsink, optstoremotion, optswitchtable, optree, optshrinkwrap, optgvnpre, optrefelide, dead_store_ext, optpure, optpartialinline, optslp, optunrolldyn, optipara, opticf, stackalloc, optfinalvalue, sibcall, optsummary, and more), including must-fire, must-not-fire, safety, and disabled-switch cases, plus per-pass codegen assertion scripts (*_check.sh) proving at the assembly level that eligible shapes fire and rejected shapes do not.
  • Verified with a stage-2 self-compile of the compiler at -O4 with all new passes enabled (including full self-hosts with the opt-in SHRINKWRAP and GVNPRE passes forced on); the full unleashed suite passes with zero regressions beyond the documented pre-existing failures.
  • Each opt-in pass is additionally verified with the switch forced on across the entire unleashed suite (identical pass/fail set to baseline). With the testtool unix run-phase fix, 1200/1202 tests pass on x86_64-linux; the two failures reproduce identically on the pre-merge compiler.

Commits (oldest first)

  • ce247f6f99 add -OoLICM loop-invariant code motion at -O4
  • f85029776e tests: optlicm loop-invariant code motion coverage
  • 5942317888 add -OoLOOPUNSWITCH loop unswitching at -O4
  • e3c1c8aca7 add -OoBITIDIOM bit-population-count idiom recognition at -O4
  • a146378dcc add -OoRANGEELIM value-range-analysis range-check elimination at -O4
  • b58a765960 add -OoVECTORIZE conservative SSE loop autovectorization
  • 87ef474c9e Signed mod by non-power-of-two constant: magic-number strength reduction
  • 612dd73ee3 add tzcnt/bsr bit-scan idiom recognition to -OoBITIDIOM at -O4
  • a417889ad1 strength-reduce loopvar*invariant multiplies inside array-index/pointer offsets
  • b4892c6230 add -OoVECTORIZE per-loop vectorization diagnostic (notes 06065/06066)
  • 68259ed54f -OoVECTORIZE: add scalar-broadcast and copy element-wise shapes
  • 9a57af019f optdeadstore: extend DSE to record-field and static-array-element stores
  • 6451f6addc aoptx86: sibling-call frame reuse for tail calls from a framed routine
  • 445fbd6c8a Add -OoJUMPTHREAD jump threading / nested re-test elimination at -O4
  • fd7296cf5b opttree: fix normalize() hoisting block-exprs out of control-flow bodies (-Oodeadstore)
  • e203bc363f compiler: make an -O4 stage-2 self-compile build clean (DFA false positives + case exhaustiveness)
  • d67e6b8af7 inline vars: infer Double for real-literal init; document Rtti unit build for tests (task 273)
  • 77a02770c5 add -OoLOOPDISTPAT loop-distribution pattern idiom recognition at -O4
  • babb3b5494 add -OoLOOPPEEL full loop peeling of small constant-trip loops at -O4
  • 971e03dd61 add -OoLOOPSPLIT loop splitting on an induction-variable crossover at -O4
  • 9293ace3fe add -OoLOOPFUSE loop fusion of two adjacent same-space loops at -O4
  • aedba022e2 add -OoIFCONVERT loop if-conversion to packed maxps/minps at -O4
  • 324f28bea6 add -OoREASSOC floating-point reduction reassociation at -O4 (fast-math gated)
  • 0e1bef5300 add -OoUNROLLJAM outer-loop unroll-and-jam at -O4
  • 53f858294c add -OoPREDCOM predictive commoning of cross-iteration reloads at -O4
  • 455f28051c add -OoSRA scalar replacement of non-escaping local aggregates at -O4
  • 10b1ba8d85 add -OoSTOREMERGE adjacent narrow-store coalescing at -O4
  • a1f1438bbf add -OoCASECLUSTER gcc-style case clustering at -O4
  • a6978eadf3 add -OoCROSSJUMP cross-jumping / tail merging at -O4
  • 8e5adcad82 add -OoBLOCKORDER static hot/cold basic-block layout at -O4
  • 776b611da0 add -OoSINK partially-dead code sinking at -O4
  • c156929359 add -OoSTOREMOTION loop store motion / scalar promotion at -O4
  • 8b4171b34c add -OoVRP interval value-range propagation with branch folding at -O4
  • ad7351332d add -OoREFELIDE managed refcount elision (opt-in)
  • a484c09620 fix REASSOC miscompile/internalerror when the counter appears in the accumulated expression
  • 1cb1ab4e9b add -OoSWITCHTABLE case-to-lookup-table conversion at -O4
  • 62aeea0f98 add -OoREE redundant sign/zero-extension elimination at -O4
  • f7f8d2c030 add -OoSHRINKWRAP shrink-wrapping (opt-in), sinking a push-only prologue below a guard clause
  • cf11e776f4 add -OoGVNPRE dominator-based global value numbering / full-redundancy elimination
  • d4645bc067 add Quick Start section to README: compiling programs and Lazarus projects with -O4
  • 0e595fe7ac add fpc.sh wrapper: invoke in-tree ppcx64 with -O4 and built unit paths
  • 5f238a038d extend -OoVECTORIZE with single-precision sum / dot-product reduction vectorization
  • e45bbf4b59 rename fpc.sh to fpcu.sh to avoid confusion with the fpc driver; document it in README Quick Start
  • b1213a0f2a drop forced -O4 from fpcu.sh so debug builds stay debuggable; adjust README
  • a7486cf488 add lazbuildu.sh wrapper: lazbuild with the in-tree compiler via fpcu.sh
  • 14f577ffa7 lazbuildu.sh: use the lazbuild on PATH instead of a sibling checkout
  • cfdeaf5297 add -OoPURE interprocedural pure/const discovery, consumed by -OoLICM
  • 1de4a6465b add -OoPARTIALINLINE partial inlining / function splitting (procedures)
  • 118bcd0684 add -OoSLP superword-level parallelism (SLP) straight-line vectorization
  • 247fba5de0 add -OoUNROLLDYN dynamic-trip loop unrolling and -OoPREFETCH software prefetch
  • 98d932ba51 extend -OoPARTIALINLINE partial inlining to functions (non-void return)
  • e9e466506d -OoGVNPRE: consume the -OoPURE verdict (CSE identical pure calls); broaden pure eligibility to const-aggregate readers
  • ce7d807420 add -OoICF identical code folding (opt-in), folding duplicate routine bodies via jump thunks
  • 4ea8de1765 add -OoSTACKALLOC escape-analysis stack allocation of non-escaping local dynamic arrays
  • 95c78053ec add -OoIPARA interprocedural register allocation (opt-in), narrowing caller-save spills to a callee's proven volatile clobber set
  • fe0a4fbbe0 add -OoFINALVALUE final value replacement + dead loop elimination (opt-in)
  • 62275fd46a shared cross-unit PPU optimizer summaries: cross-unit -OoPURE + -OoIPARA
  • 39786cb38c -OoICF: symbol-alias mode + cross-unit identical code folding
  • c2a48b0437 -OoVECTORIZE: double-precision loop autovectorization (128-bit SSE/AVX)
  • 7ac3c112fe -OoPURE: -vh purity hints + read-only method/accessor eligibility
  • e620c046cc Fix cs_opt_level2 const-fold spurious overflow on internal wraparound
  • 4bf0beabd2 add -OoSIBCALL sibling-call optimization (opt-in)
  • a7a109c1bf -OoFINALVALUE: pointer-stride accumulators + multi-statement bodies
  • 26da550200 -OoPURE: treat pure/const calls as non-barriers in extended dead-store elimination
  • ad3964f734 optvectorize: fix -Oodeadstore x -OoVECTORIZE reduction miscompile (stale accumulator seed)
  • f7be469997 -OoFINALVALUE follow-ups: 64-bit counters, -Cr range-check regime, static/global accumulators
  • 5898a20945 -OoSIBCALL: multi-location result forwarding + frame-address escape gate
  • 1f5e957d97 fixup after cherry-picking main optimizer work: satisfy un_main's stricter case-exhaustiveness check in optloop.pas
  • c95bce2901 testtool: fix run phase on unix targets

fibodevy added 30 commits May 23, 2026 11:07
`SizeOf(record.field)` and `BitSizeOf(record.field)` now return the
slot the field actually occupies in the surrounding record when a
`size N` override is in effect, matching what `OffsetOf` of the next
field implies. before, both fell back to the declared type's natural
size while the slot was silently padded under them - reading either
intrinsic gave a value that did not match `OffsetOf` deltas.

mirrors the existing `bitsize N` handling (`BitSizeOf` already
honoured `custom_bitsize`). pure type queries (`SizeOf(TypeName)`,
`SizeOf(EnumConstant)`) are unchanged - they have no field context.
…ngword

`parsed_custom_align` / `parsed_custom_bitalign` in pdecvar.pas and the
matching `custom_align` / `custom_bitalign` fields on tfieldvarsym +
tfield_sizing_entry change from shortint to longword. PPU read/write
switches from byte to longint to match. `alignrecord` parameter and the
varalign / varalignrecord locales in symtable.pas widen to longint so
they stop wrapping at 128. parse_modern_union's `target_align` widens to
longword and gets a high(shortint) cap before assigning into the
recordalignment field (which stays shortint - stock FPC infrastructure).

Field-level `align N` now accepts powers of two up to 128, 256, 1024
etc. without the shortint wraparound that previously corrupted the
sign bit and tripped the `<1` validation. The shortint
recordalignment field still caps record-level alignment at 127, so
`union align 128` / `record align 128` clamp at 127 rather than
overflow. Lifting that cap is a separate change to the stock FPC
recordalignment PPU format.
`ncipollo/release-action@v1` with `allowUpdates: true` plus
`richardsimko/update-tag@v1` worked only when nothing touched the
release between runs. Manually deleting the release, renaming its tag,
or moving artifacts left the action in a state it couldn't recover from
and the next run failed instead of recreating.

`gh release delete --cleanup-tag` followed by `gh release create` is
state-independent: whatever the upstream looks like, the run lands at a
predictable result. Also resets `published_at` on every build so the
release no longer shows a stale "released N days ago" timestamp.
…n enum field

An enum-typed record field can only carry the `of T` storage-type
modifier, not `size N`. The compiler now rejects the combination with
a dedicated error message. Drop the override from `TE.kind` so the
test exercises only the legal `of Byte` form, and adjust the matching
halts for the natural 1-byte slot size.
equal_signature in pparautl wants te_exact for both params and returndef
to bind a method implementation to its forward declaration. The two
`(Integer, Integer)` parses produce separate recorddefs that only matched
as te_equal, so `function tt.GetPair: (Integer, Integer)` failed to bind
to its in-class declaration and the compiler reported the method as a
new overload with the same parameter list.
Forward and impl both with anonymous return (`function f: array of string;`)
create distinct tdef instances. Pointer-equality compare rejected the
impl. Now equal_signature re-points the impl's returndef + hidden
funcret paravarsym at the forward's def when both sides have no
typesym and are structurally equal.
composable_records PPUs carry recorddef sections with nested enumdefs;
loading them in non-composable mode hit insertdef with defowner still
nil, since trecorddef.ppuload assigns defowner only after the inner
symtable finishes loading.
Adds `m_implicit_generics` to `unleashedmodeswitches`. Existing tests
that used the explicit `generic` / `specialize` keywords (which the
parser only consumes when implicitgenerics is OFF) drop those prefixes.
Per-unit `{$modeswitch striprtti}` and the `-Mstriprtti` mode-prefix
both reset on any `{$mode X}` directive that follows them. Add a
dedicated global CLI flag `--striprtti` stored as
`force_striprtti_cli` in globals; `rtti_string` short-circuits when
either the modeswitch is active OR the global flag is on. Existing
whitelisting (`expose`, `{$rttiexpose}`, `--rttiexpose=`) keeps
working since the early-return check stays before the whitelist
exits.
`[(1)/100]` failed with `"]" expected but "/" found` because the
tuple-aware path in `factor_read_set` consumed `(expr)` itself and
swallowed the closing `)`, leaving no one to parse the `/100` tail.
factor's own `_LKLAMMER` handler already detects tuple literals at
the right nesting level - just delegate to `comp_expr`.
multi-dim and mixing with ranges come free via the comma loop:
array[3, 'a'..'z'] becomes array[0..2, 'a'..'z']. N must be a
positive integer constant.
Switches the bullet list to a 4-column table covering everything in
extra-improvements.md - including compound `+=` and `inc` / `dec` on
properties, which the bullets had skipped - and adds the `array[N]`
shorthand to the docs/README.md catch-all paragraph.
capture-detection and the capturer-chain skipped blocksymtable, so a
closure read the var on its own frame as garbage and outer writes
never reached the capturer field. blocksymtable now anchors defowner
and symtablelevel at the enclosing procdef rather than the immediate
parent (which may be a with-record fieldset or `on E` exceptsymtable
whose levels and defowners are unrelated).
Pulling in SysUtils just for its Swap<T>/Exchange<T> generics drags in
exception and handler setup; SwapValues is a System-only builtin. Gated on
{$mode unleashed} and only when no SwapValues symbol is in scope, so it
cannot shadow existing code.
after an anonymous function is reparented into a capturer method (normal_function_level) its block symtables keep the nested level they were anchored to at parse time. a managed inline var in a nested begin..end then had its implicit finalization treated as a parent-frame access: the blocksymtable load saw symtablelevel <> current level and called set_needs_parentfp on a level-2 method, giving IE 2020050302. unmanaged block vars have no synthesized finalization load, which is why only managed types tripped it.

guard the parentfp branch on defowner identity: a block var whose blocksymtable belongs to the current procdef lives on the current frame, so skip the parent-fp load. genuine captures (defowner is an outer proc) are unaffected, so b47656b still holds. adds anon_managed_blockvar_01 regression test.
…inator

append_else walks the if-chain assuming every `t1` is another ifn; once
a `_:` branch deposited a non-ifn tail, a trailing else/otherwise
dereferenced past it. Reject the redundancy with an error instead. In
expression mode `else <expr>` or `otherwise <expr>` terminates the
match itself - the explicit `end` is only required when there is no
else/otherwise clause.
Previously emitted `Syntax error, "ELSE" expected but "END" found`
which doesn't mention that `otherwise` is also accepted.
When the host compiler is x86_64 without `FPC_HAS_TYPE_EXTENDED` (e.g.
self-built on win64), the assembler writer in `assemble.pas` rejects
80-bit float constants with IE 2015030501. The existing safeguard only
covers i386 cross-compiles. Auto-define `FPC_SOFT_FPUX80` here so the
soft x80 emulation path is taken whenever the host can't natively
represent extended.
Mirror of i386-nativent for x86_64, registered as `system_x86_64_nativent`
(#127). Sysinfo uses Win64-style settings: `cpu_x86_64`, `as_x86_64_pecoff`,
16-byte stack/parm alignment, x86_64 llvm datalayout. Linker entry names
keep the underscore prefix (`_NtDriverEntry` / `_NtProcessStartup` /
`_DLLMainStartup`), matching how win64 RTL declares its CRT startups.

Treated like win64 for fpu types: `pbestrealtype` -> `s64floattype`,
no separate `Comp` type, no `FPC_HAS_TYPE_EXTENDED`. This keeps the
generated code free of legacy x87 80-bit operations that the win64 ABI
also avoids.

Target is flagged `tf_under_development` (upstream FPC has only the
i386 variant); NDK / DDK header structs are still i386-shaped, so
runtime correctness on real x86_64 NT is not guaranteed yet.
Add `x86_64-nativent` to `MAKEFILETARGETS` and duplicate the
target-specific blocks (`TARGET_UNITS`, `TARGET_IMPLICITUNITS`,
`TARGET_RSTS`, `COMPILER_INCLUDEDIR`, `COMPILER_SOURCEDIR`). Produces
79 .ppu (system, sysutils, classes, ndk, ddk and the rest).
Flip the `nativent` row in `OSCpuPossible` so `x86_64` is allowed
alongside `i386`. Without this `fpcmake` refuses to regenerate
Makefiles for `x86_64-nativent` and `make install` fails before it
reaches the compile stage.

`rtl/Makefile` is the regenerated artifact: `x86_64-nativent` joins
`MAKEFILETARGETS` and the `TARGET_DIRS+=nativent` / `TARGET_DIRS_NATIVENT`
blocks pick up the new pair.
PE32+ format requires 8-byte IAT/INT thunks (ULONGLONG per entry).
The set `systems_peoptplus` gates that behaviour in `ogcoff.pas`
(import generation), `assemble.pas` (aitconst_rva_symbol width)
and `aasmtai.pas` (rva symbol size). x86_64-nativent was missing,
so the linker emitted 4-byte thunks and `.idata` parsed by any PE
viewer collapsed into duplicated names and "<corrupt>" tails - the
upper half of each 8-byte slot got read as a separate 4-byte entry.

After this driver imports parse cleanly:
  ExAllocatePoolWithTag, ExFreePoolWithTag, DbgPrint.
Driver code generated for x86_64-nativent was passing arguments via
RDI/RSI/RDX/RCX/R8/R9 (SysV) while the Windows kernel calls drivers
with the MS x64 convention (RCX/RDX/R8/R9 + 32-byte shadow space).
The first argument to `DriverEntry` arrived in RDI, but the body
read it from RCX -> kernel followed a garbage `DRIVER_OBJECT*` and
the box froze on load.

Introduce `systems_x86_64_ms_abi = [system_x86_64_win64,system_x86_64_nativent]`
and route the ABI checks in `cgcpu.pas` (prologue/epilogue, shadow
space, XMM saves) and `cpupi.pas` (`x86_64_use_ms_abi`,
`set_first_temp_offset`, `generate_parameter_info`) through it.

SEH64 (table-based unwind in nx64flw.pas) stays gated on
`system_x86_64_win64` alone - the nativent RTL has no `_fpc_local_unwind`
yet, so emitting Win64 unwind directives there would break linking.
joaopauloschuler and others added 18 commits July 9, 2026 15:06
…oaden pure eligibility to const-aggregate readers

Two follow-ups to the landed -OoPURE interprocedural pure/const discovery:

* -OoGVNPRE now value-numbers calls to routines proven PURE by -OoPURE. A
  direct, non-method call to a pure routine (reads but never writes global
  state, no I/O, cannot raise/trap) whose arguments are side-effect-free is
  treated as a memory-reading GVN candidate, keyed on its argument's safe
  locals. Two structurally-identical such calls with the arguments and the
  pure-read memory unchanged in between are commoned to a single call reused
  from a temp. A pure call is no longer a straight-line barrier (new
  gvn_has_sideeffects), so `x := f(a)` participates; its arguments are still
  scanned so a side effect hiding in one re-arms the barrier. Kill semantics
  ride the existing gvn_mem machinery: any store to memory, an impure call, or
  a loop that writes memory invalidates the value. Gated on -OoPURE; inert
  when the pure pass did not run.

* -OoPURE eligibility broadened: a routine that reads through a const/constref
  aggregate parameter (unmanaged record / fixed array / set) but never writes
  it is now a PURE reader -- its field/element reads are pure memory reads and a
  const parameter cannot be written. Result type stays a simple scalar and
  methods (a mutable self alias) remain out of scope, keeping the gates
  conservative. These are pure-not-const, so they feed the new GVN-PRE CSE
  (LICM, which needs const, is unaffected).

Tests: optgvnpre_purecall_01 (pure-call CSE correctness oracle: full/partial
redundancy plus kill-on-arg-write and kill-on-global-store soundness),
optgvnpre_purecall_disabled_01 (disabled-switch parity), optpure_constaggr_01
(const record/array reader eligibility, memory-kill soundness, var-aggregate
stays impure). Full unleashed suite green (968 pass / 8 known-skip / 0 fail);
unrolldyn_check.sh and slp_codegen_check.sh still pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… bodies via jump thunks

Ports the gcc -fipa-icf / gold --icf idea to FPC as an opt-in -OoICF pass that
runs intra-unit at the assembler-list level, after every routine of a module
has been generated into al_procedures and before the object file is emitted
(hook in create_objectfile, pmodules.pas).

For each routine it canonicalizes the body (opcode/opsize/condition, every
operand encoding, memory references with base/index/scale/segment/offset/
refaddr) with the routine's own symbol and local labels symbolized to
positional tokens and other relocations abstracted to the referenced symbol
name. Routines with equal canonical signatures emit byte-identical machine
code, so the first is kept and each later duplicate has its instruction body
replaced by a single `jmp <kept>` thunk.

The fold is always a thunk, never a symbol alias: every duplicate keeps its own
distinct symbol and therefore its own distinct address, so @f<>@g survives even
when a folded routine's address is taken and compared, and FPC's non-DWARF
exception model is unaffected. All labels are preserved so the already-generated
debug/CFI list never dangles. Conservative: only straight-line bodies (no
embedded data/cfi/unhandled operand kinds) fold, and only when the routine has
enough instructions that a 5-byte thunk is a net shrink. x86 only; a no-op
elsewhere. Opt-in; NOT part of the -O4 defaults.

New cs_opt_icf switch (globtype.pas) + 'ICF' name + ppudump table entry.
Tests under unleashed/tests/testfiles/opticf: two identical routines fold (a
runtime probe asserts exactly one begins with the 0xE9 jmp opcode) while a
near-identical routine does not; addresses stay pairwise distinct; results match
ICF on vs off; and a generics-instantiation case. Full suite green (972 pass, 7
pre-existing ignores) both normally and with -OoICF forced on globally;
unrolldyn_check.sh and slp_codegen_check.sh still pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…cal dynamic arrays

Intraprocedural escape analysis that proves a local dynamic-array variable
never outlives its routine and replaces its heap buffer with a frame-local
stack buffer, eliding the fpc_getmem/fpc_dynarray_setlength call. This is the
HotSpot / gcc -fipa-pta stack-allocation idea, the safe dynamic-array subset.

A local dynamic-array var A of a NON-managed element type qualifies when its
ONLY SetLength is SetLength(A,N) with N a positive compile-time constant and
header+payload within a 4 KiB frame budget, and A provably never escapes:
it appears only as A[i], as the operand of Length/High/Low or of that one
SetLength, is never address-taken, captured by a nested routine, assigned
to/from another location, or passed whole to a callee. Recursive routines,
routines using exceptions and assembler routines are disqualified.

The rewrite runs BEFORE do_firstpass (next to -OoREFELIDE), while SetLength is
still an in_setlength_x inline node and A[i]/Length are plain nodes. It
allocates a hidden local record  record refcount,high:sizeint; data:array[
0..N-1] of elem end  and lowers the SetLength (guarded by  if A=nil  so loop
re-execution stays the RTL's SetLength-to-same-length no-op) to:

    FillChar(R,sizeof(R),0);  R.refcount:=-1;  R.high:=N-1;
    PPtrUInt(@A)^ := PtrUInt(@R.data);

refcount:=-1 is exactly FPC's own constant/read-only dynarray header sentinel
(aasmcnst begin_dynarray_const), so the scope-end finalization the compiler
still emits for A (fpc_dynarray_decr_ref) neither frees nor finalizes it, and
fpc_dynarray_incr_ref treats it as constant. The stack buffer has precisely
the heap buffer's lifetime, so behaviour is bit-identical.

Follows the established fork pattern: cs_opt_stackalloc in globtype.pas (+
OptimizerSwitchStr and ppudump table), OptimizeStackAlloc in optloop.pas,
call site in psub.pas. Opt-in (-OoSTACKALLOC); NOT part of the -O4 defaults.

Tests: unleashed/tests/stackalloc_codegen_check.sh asserts the non-escaping
array emits no allocator call (FILLCHAR instead) while a returned one still
allocates; testfiles/stackalloc/ covers non-escape zero-fill/index/Length
semantics, the loop-reexecution guard, each escape route (returned, stored to
global, aliased) staying heap-allocated and correct, managed-element arrays
not transformed, and by-var element passing. Full suite 978/985 (only the 7
known pre-existing failures) both normally and with -OoSTACKALLOC forced on
globally; unrolldyn_check.sh and slp_codegen_check.sh still pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…caller-save spills to a callee's proven volatile clobber set

Ports the idea of gcc's -fipa-ra to FPC, intra-unit and x86_64 only. When a
routine's code is generated its body is already register-allocated, so the
register allocator knows exactly which physical registers it uses
(rg[bank].used_in_proc -- the same set FPC uses to pick prologue callee-saved
pushes). optipara.RecordProcClobbers snapshots, per register bank, the
intersection of that set with the ABI volatile (caller-saved) mask and records
it on the tprocdef. That summary is already transitively complete: every inner
call allocates its callee's clobbers around itself, so a routine calling an
un-analysed target ends up with the full mask (no unsafe reduction downstream);
one that only calls small proven leaves inherits just their small sets.

Consumer side (tcgcallnode.pass_generate_code): for a DIRECT call to an
already-recorded routine, the volatile registers allocated around the call are
narrowed to the callee's clobber set, so caller values living in untouched
volatile registers survive the call without a spill/reload. On a small-leaf hot
loop this removes the rbx/r12/r13/r14 evacuation entirely.

Both integer and XMM/MM clobbers are tracked. Conservative fallbacks to the full
ABI mask (a too-small set silently corrupts caller state):
  - indirect/procvar, virtual, method-pointer, external, syscall, interrupt and
    assembler callees;
  - routines not yet generated in this compilation (forward order);
  - inline-asm-containing or exception-using callees (marked ipara_full);
  - and, crucially, any call inside a caller that itself has exception handling
    or an implicit finally -- an exception unwind (longjmp) restores only
    callee-saved registers, so a caller value kept in a volatile register across
    a call that raises and is caught here would be lost.

The per-procdef summary is transient (never written to / read from the ppu), so
on load a routine defaults to "not analysed" => callers use the full ABI mask.

Gated behind -OoIPARA; NOT added to -O4 this round. Adds the cs_opt_ipara switch
(globtype OptimizerSwitchStr + ppudump table), an ipara_codegen_check.sh
assertion and runtime regression tests under testfiles/optipara/ (int leaves,
XMM leaves, and every fallback path). Full unleashed suite: no new failures with
the switch off or forced on globally.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t-in)

Port of gcc's -ftree-scev-cprop (scalar-evolution final-value replacement)
plus the whole-loop deletion its control-dependent DCE performs. New optimizer
unit compiler/optfinalvalue.pas, wired as opt-in -OoFINALVALUE (globtype.pas
enum + name table, ppudump.pp mirror, psub.pas call site before the DFA/loop
passes so the still-structured for-nodes' bounds are read directly). NOT in the
-O4 defaults.

Transform: a counted for-loop whose whole body is a single loop-invariant
accumulator update of a plain local integer (inc(s,c) / dec / s:=s+c / s:=s-c),
whose counter's exit value is dead, is replaced by the closed form of s's exit
value  if a<=b then s:=s+(b-a+1)*c  (to<=from for downto) and the loop deleted;
an empty-bodied dead counted loop is deleted outright. Post-loop uses of s then
read the closed form directly.

Sound conservative subset (correctness over coverage): integer inductions only;
counter a plain local integer <=32 bits (so the symbolic trip count b-a+1 is
exact in 64-bit); accumulator a plain local integer distinct from the counter;
c and the bounds loop-invariant, side-effect-free and independent of i/s;
counter/accumulator rejected if address-taken, captured by a nested scope or
volatile; zero-trip loops handled by the a<=b guard; two's-complement
wraparound preserved so the result is bit-identical, hence DISABLED under
-Co/-Cr; break/continue/exit/call/store bodies never match.

Tests: unleashed/tests/testfiles/optfinalvalue/ (ascending/downto/zero-trip
accumulators, byte/word/shortint wraparound, invariant step, empty-loop
deletion, break/global/live-counter/nested-capture/addr-taken no-transform
cases, switch round-trip, -Co disable) all self-checking bit-exact vs a direct
reference; finalvalue_check.sh proves at the assembly level that eligible loops
lose their back-edge while rejected loops keep it, and that ON/OFF output is
identical. Full unleashed suite: 991 pass / 6 known-ignored, both normally and
with -OoFINALVALUE forced on every test; ipara/slp/stackalloc/unrolldyn checks
still pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add one generic per-procdef "optimizer summary" blob persisted through the PPU,
then consume it for cross-unit -OoPURE (pure/const CSE of used-unit calls) and
cross-unit -OoIPARA (narrowing caller-save spills around a used-unit leaf call).

Mechanism (compiler/symdef.pas tprocdef.write_optimizer_summary /
load_optimizer_summary): a self-describing (tag,len,payload) list streamed
inside the ibprocdef entry, terminated by optsum_end. An older/absent summary
yields no tags -> the existing conservative fallback. Tags live in ppu.pas
(optsum_pure/optsum_ipara, optsum_icf reserved). Framing is versioned once via
CurrentPPULongVersion 30 -> 31; new per-pass payloads are just new tags.

- PURE: the DERIVED pure/const verdict is written as two booleans (computed at
  write time via a hook optpure registers, so the whole defining unit is already
  analysed). A used-unit routine carries pure_ppu_valid + the verdict; dfs_pure
  treats it as a leaf with that verdict. Kept in the interface crc so a flipped
  verdict recompiles dependents.
- IPARA: the volatile-register clobber mask is written guarded by a
  target/ABI/-Cf instruction-set signature; a mismatching or full summary is
  dropped to the full ABI mask. The ncgcal consumer already had no same-unit
  gate, so cross-unit narrowing works once the mask is loaded.
- ppudump dumps the new blob (PURE verdict, IPARA signature + mask, unknown tags
  skipped by length).

Tests: unleashed/tests/optsummary/ two-unit fixtures + optsummary_check.sh
(cross-unit pure call CSEs 3->1 under -OoGVNPRE -OoPURE, absent summary keeps 3,
bit-exact/correct in every config); ipara_codegen_check.sh extended with a
cross-unit case (used-unit leaf narrows callee-saved pushes 4->0, falls back
when the callee unit lacks the switch). Full unleashed suite green normally and
with -OoPURE -OoIPARA -OoGVNPRE forced on (992 pass / 0 unexpected / 5 ignored);
RTL rebuilt for the version bump; all *_check.sh pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two follow-ups to the intra-unit -OoICF pass (compiler/opticf.pas):

1. Symbol-alias mode. A proven address-never-observed duplicate is now
   collapsed to a *zero-byte* symbol alias (its entry symbol emitted as an
   additional label at the survivor's address, the same two-labels-one-location
   mechanism the codegen already uses for main/PASCALMAIN) instead of a 5-byte
   jmp thunk. Gated conservatively via a new source-level address-taken proof:
     - tprocdef.icf_addrtaken, set at the canonical proc->procvar conversion
       (ttypeconvnode.create_proc_to_procvar) and at tloadnode.create_procvar,
       captures every @proc / procvar assignment / procvar argument;
     - the routine must be source-invisible to other units (procsym in the
       static/local symtable, not exported/public/external/virtual, not a
       method), so no source anywhere can form @dup;
     - single entry symbol only (multi-aliasname routines fall back to thunk).
   Address-taken / interface-visible / method duplicates keep the
   address-preserving thunk, so @f<>@g stays observable where guaranteed. We do
   NOT gate on the ELF global bind: under default section-per-symbol
   smartlinking every routine symbol is emitted global, yet an
   implementation-section routine is still un-nameable from another unit.

2. Cross-unit folding via the shared PPU optimizer-summary mechanism. Each
   globally-visible routine that survives intra-unit folding stores a 128-bit
   digest of its full canonical instruction stream on its tprocdef, persisted as
   the (previously reserved) optsum_icf ppu tag guarded by the IPARA-style
   target/ABI/-Cf signature and carrying the survivor's global name (captured at
   write time; mangledname is not yet resolvable at load). A later unit registers
   every used unit's survivors and folds a digest-matching routine into a jmp
   thunk to that external global symbol. Cross-unit folds are always thunks
   (never aliases): the survivor lives in another object and a thunk keeps every
   duplicate's own address intact. Collision stance: cross-unit we trust the
   128-bit digest of the full canon alone (no body to compare) -- birthday-bound
   ~n^2/2^128, and the identical canonicalisation + signature guard must also
   match; documented in the unit header. The intra-unit bucket now keys on the
   digest and still byte-verifies via full canon compare (fixing latent
   truncation of >255-char canons, since mangled names are shortstrings here).

No PPU version bump (new length-prefixed, skippable summary tag). Survivors are
kept alive under smartlinking by the thunk/alias reference; verified with
default flags and -XX -CX -Xs.

Tests: unleashed/tests/testfiles/opticf/opticf_alias_01.pp (alias correctness);
unleashed/tests/icf_check.sh + icf/ fixtures (zero-byte alias asm shape,
address-taken thunk fallback with @ping<>@pong, cross-unit external jmp,
non-matching bodies never fold, bit-exact vs ICF-off and under smartlink).
Full unleashed suite: 992 pass / 0 unexpected. With -OoICF forced on: only
opticf_disabled_01 "fails" (it asserts ICF-off, so forcing it on is by design).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extend the -OoVECTORIZE loop autovectorizer from single-only to also handle
`double` dynamic arrays, 2 lanes per 128-bit window. Every existing shape now
fires on double: array-array a[i]:=b[i] op c[i] (+ - *), scalar-broadcast
a[i]:=b[i] op s / s op b[i] (incl. non-commutative subtraction and constant
literal), plain copy a[i]:=b[i], and the sum / dot-product reductions
(including the FMA-contracted form) under fast-math.

Backend: tvectoropnode gains an `isdouble` flag (nbas.pas); the x86 emitter
(nx86inl.pas) selects movupd/addpd/subpd/mulpd, movsd+unpcklpd for the splat,
xorpd+movsd for the reduction seed, and an unpckhpd+addsd 2-lane horizontal
sum, with the VEX v-forms under an AVX fputype (still 128-bit). min/max
if-conversion stays single-precision only.

Recognizer (optloop.pas): the precision is fixed from the destination array
(store shapes) or the accumulator (reductions); every source array, scalar
operand and RHS/addend must match it, so any single/double mix is rejected and
stays scalar (a wrong lane width would miscompile). All single-precision
soundness gates are preserved unchanged: plain-counter index, -Cr/-Co disable,
globals/address-taken/volatile scalar rejection, self-alias safety, bit-exact
vs the scalar path (incl. NaN/Inf/-0.0 and tail residues), and the fast-math
gate on reductions. The SLP straight-line vectorizer stays single-only.

AVX-256 (ymm) widening is intentionally NOT taken here: it needs a 256-bit
horizontal-reduction path and OS_M256 codegen across the node, which is
invasive/higher-risk; left as a documented follow-up.

Tests: unleashed/tests/testfiles/optvect/vect_double_* (10 runtime bit-exact
tests mirroring the single ones: per-op array-array, scalar-broadcast, copy,
sum+dot reductions, store & reduction special-values, self-alias, mixed-type
rejection, -Cr disable, AVX) plus vectorize_double_check.sh (asserts double
loops pack to addpd/movupd, stay scalar with the switch off, and that a mixed
single/double loop never vectorizes). Full unleashed suite green normally and
with -OoVECTORIZE forced on (1000 pass / 0 unexpected).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(d) Emit a -vh hint per analyzed routine when -OoPURE discovers it
pure/const (messages 06087/06088, once at AnalyzeProcPurity time, gated
by hint verbosity). Reports the strongest verdict (const implies pure).

(e) Broaden purity eligibility to read-only instance/class methods (the
classic  function TFoo.GetX:Integer;begin Result:=FX end  accessor):
 - proc_eligible now accepts non-virtual, non-abstract, non-ctor/dtor
   methods; self is skipped as a read-only reference parameter.
 - lvalue_write_is_side_effect treats ANY store reachable through self as
   a side effect regardless of how self is passed (a by-value class
   pointer or a by-ref record self), so a method that writes a field is
   correctly impure -- the soundness-critical gate.
 - A method reads self's referenced memory, so it is at best "pure",
   never "const" (pure_reads_global stays set). No new PPU summary flavour
   is needed: the "pure" bit already means memory-dependent, and the
   -OoGVNPRE consumer treats every pure call as a memory reader.

-OoGVNPRE method-call CSE (memory dependence): gvn_pure_call now accepts
a provably-direct pure method call (non-virtual/abstract procdef,
side-effect-free methodpointer); gvn_candidate value-numbers it as
gvn_mem keyed on the locals read by self AND the arguments. Any
intervening store (including a field write to the object) or any call
therefore invalidates it -- two getter calls with a field write or an
impure call between them are NOT commoned; two with nothing between are.
Virtual/abstract methods are never folded. Cross-unit works via the
existing optsum_pure bit (a pure method serializes exactly like a pure
function).

Tests: optpure_method_01.pp (bit-exact runtime: record-self getter
commons, class getter with intervening field write / call / object
reassignment does not); pure_method_check.sh (codegen call-count
assertions incl. virtual-not-folded); pure_hint_check.sh (verdict text,
silent for impure and with -OoPURE off). Full unleashed suite 1003 pass /
0 unexpected, bit-clean normally and with -OoPURE -OoGVNPRE forced on;
all *_check.sh green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The level2 range-test optimization in nadd.pas (is_range_test) rewrites
  (C1 <= expr) and (expr < C2)
into the unsigned test
  (expr - C1) <= (C2 - C1).
When the lower bound C1 is negative, the low-bound constant is stored as
an nf_internal *unsigned* ordinal (adaptrange converts e.g. -149 to
qword(-149) = $FFFFFFFFFFFFFF6B). When such a helper is inlined at a call
site where "expr" folds to a compile-time constant, the const-fold in
taddnode.simplify evaluated e.g.
  0 - qword(-149)
which sets the tconstexprint overflow flag (0 < $FF..F6B in unsigned
arithmetic) and raised a spurious user-facing
"Overflow in arithmetic operation" ERROR at any level enabling
cs_opt_level2 (-O2/-O3/-O4) -- even though this is intentional
modulo-2^n wraparound in compiler-generated code.

This made pletora/neural-api's neuralvolume.pas uncompilable at -O2+:
inside pcr_powf -> is_exact_pf the range test
"(-149 <= Trunc(y)*e) and (...)" folded once y=1/2.4 gave Trunc(y)=0,
and the error surfaced (via the inlined body's stored fileinfo) at
lab2rgb's three float "/const" statements, neuralvolume.pas(2147,1).

Fix: in the addn/subn/muln integer const-fold, do not raise
parser_e_arithmetic_operation_overflow for nf_internal nodes; keep the
wrapped value (create_simplified_ord_const masks it to the result type).
User-level overflow reporting is unchanged for non-internal nodes.

Regression test const_fold_range_test_neg_lowbound_01 fails to compile
at -O2 with the fix reverted (clean at -O1) and compiles+runs correctly
with the fix.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Port gcc's -foptimize-sibling-calls as an opt-in -OoSIBCALL pass, distinct from
the self-recursion loop rewrite (-OoTAILREC/cs_opt_tailrecursion) and from
partial inlining. When a routine's last action is a direct tail call to ANOTHER
routine (result := OtherFunc(args) or a bare tail procedure call), the frame
teardown is hoisted above the call and the call is rewritten to a jmp, so the
callee reuses the caller's return slot: O(depth) stack becomes O(1) for mutually
recursive pairs and continuation-style dispatch.

Implemented at the x86-64 assembler-peephole level (TX86AsmOptimizer.OptSibCall,
run from PostPeepholeOptCall, gated on cs_opt_sibcall). It also handles the
result-forwarded-through-a-callee-saved-register shape that the -O4
CallFrameRet2Jmp deliberately punts on -- the shape the register allocator emits
for `result := OtherFunc(args)`, i.e. exactly the mutual-recursion case:
recognises  call X / mov %rax,%cs / [shared label] / mov %cs,%rax / <rsp
release + callee-saved pops> / ret, hoists only the pops+release (never the
result movs -- the jmp leaves the result in rax), and jmps to X.

Sound subset (x86-64 SysV/Linux only; anything not provably safe falls back to a
plain call):
  * direct call to a symbol (indirect/procvar rejected);
  * teardown = leaq/addq rsp release and/or pops of callee-saved integer regs;
  * optional single result forward rax->callee-saved->rax (identity);
  * no outgoing stack arguments in the routine (maxpushedparasize=0), so a
    callee whose stack-arg area exceeds our zero incoming area is excluded;
  * caller convention register/cdecl/stdcall (safecall + exotic excluded);
  * CurrentProcAllowsSiblingTailFrameReuse: no try/finally/exception frame, no
    dynamic stack alloc, no nested-frame capture, no address-taken local/param
    (so no @Local can escape into the callee's arguments).
Opt-in; NOT added to genericlevel4optimizerswitches.

Switch wired in globtype.pas (cs_opt_sibcall + 'SIBCALL') and the ppudump name
table; -Oo parsing is table-driven.

Tests: testfiles/sibcall/ (even/odd + continuation runtime correctness, and a
disqualifier correctness file) pass under the suite both normally and with
-OoSIBCALL forced on the whole suite (1006 pass / 6 known pre-existing fails,
unchanged either way). unleashed/tests/sibcall_check.sh asserts the jmp in the
eligible case, a plain call with the switch off, and a plain call for each of
the four disqualifiers. An even/odd pair to depth 10^7 completes under
ulimit -s 512 with the flag and segfaults without it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extends the opt-in final-value-replacement / dead-loop-elimination pass with
two sound follow-ups:

* POINTER-STRIDE accumulators: a counted loop whose body advances a plain
  local pointer by a loop-invariant stride via inc(p,stride)/dec(p,stride)/
  inc(p) is replaced by the closed form p := p + (b-a+1)*stride, built as a
  pointer+integer add so the compiler applies the same element-size scaling
  the loop body did. The still-unlowered inc/dec inline form carries the raw
  (unscaled) element stride; the p:=p+stride assignment form is deliberately
  rejected because by this point the stride is already element-size-scaled and
  rebuilding a pointer+integer closed form would scale it a second time.

* MULTI-STATEMENT bodies: a body of several statements, each an independent
  accumulator update of a DISTINCT plain local (integer or pointer), matches --
  one closed form per accumulator under the shared trip-count guard. Two updates
  of the same accumulator (double-count) reject the loop.

Every increment must be loop-invariant and reference neither the counter nor
any accumulator, so the accumulators evolve independently and statement order
is irrelevant. All prior soundness gates (<=32-bit counter, dead counter, no
overflow/range checking, zero-trip guard, two's-complement wraparound) are
preserved. Still opt-in; not in the -O4 defaults.

Tests: finalvalue_check.sh grows codegen assertions (pointer-inc and multi-
accumulator loops deleted; duplicate-accumulator and pointer-assign loops
kept). New runtime fixtures fv_pointer_stride_01 (typed + PChar pointers, unit
and non-unit strides, inc/dec, ascending/downto, zero-trip) and fv_multi_accum_01
(two/three integer accumulators, mixed integer+pointer body), plus dup-accumulator
and pointer-assign kept-loop cases in fv_no_transform_01, all checked bit-exact
against a direct reference and ON/OFF byte-identical.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e elimination

The extended (record-field / static-array-element) DSE in optdeadstore.pas
treated every call as a hard barrier that flushed the whole pending-store set.
With -OoPURE on, a call whose resolved direct target carries a pure or const
verdict is now relaxed, precise about the pure-vs-const asymmetry:

* const call: reads and writes no memory -> full non-barrier; pending stores
  survive (only its argument expressions, scanned normally, perform reads);
* pure call: writes nothing but MAY READ global/heap state -> treated as a
  potential reader: pending stores to globally reachable slots (static vars,
  by-ref const params) are kept live, while stores to non-address-taken locals
  and by-value params (which no callee can name) still survive;
* impure / indirect / procvar / virtual / aggregate-returning calls remain
  full barriers, as do deref and asm nodes. Soundness first.

Both the non-candidate-assignment scan and the previously unconditional
"any other statement" barrier for bare call statements go through the new
relaxed path (el_classify_call / el_haz_scan / apply_relaxed_barrier).

Tests: unleashed/tests/pure_dse_check.sh asserts on codegen that the dead
store IS removed across a const call and across a pure call on a local base,
and is NOT removed across an impure call or across a pure call on a static
base, and that the baseline (no -OoPURE) keeps everything; runtime fixture
testfiles/optpure/optpure_dse_01.pp is deterministic and self-checking at
-O1/-O2/-O3+flags/-O4+flags. Full suite 1009 pass / 6 known pre-existing
fails normally AND with -OoPURE forced suite-wide; all 12 *_check.sh PASS.
(The 7 optvect failures under forced -Oodeadstore are a pre-existing
-Oodeadstore+VECTORIZE-reduction interaction, reproduced with this change
stashed; not introduced here.)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tale accumulator seed)

Root cause
----------
The autovectorizer's reduction path (OptimizeVectorize, vok_reduce_sum /
vok_reduce_dot in optloop.pas) seeds lane 0 of the packed accumulator with the
incoming scalar accumulator s -- i.e. the user's  s := <start>  store just
before the loop. That seed read of s was embedded directly inside the
create_reduce_init backend (tvectoropnode) node:

    acc := reduce_init( acctemp, load(s) )

A backend node's operand reads are not reliably modelled by data-flow analysis
(the finish/horizontal-sum store already worked around this by writing s with a
plain assignment from a temp, per the comment at the store site). Because DFA
did not see s being read at the seed, the incoming  s := <start>  def looked
dead, and with dead-store elimination enabled (-Oodeadstore) the stock DSE pass
in optdeadstore.pas removed it. Lane 0 was then seeded from a stale stack slot,
producing a wrong sum / dot-product result.

This is why forcing -Oodeadstore suite-wide made 7 optvect runtime tests fail
(vect_reduce_01, vect_reduce_avx_01, vect_reduce_disabled_01,
vect_reduce_special_values_01, vect_double_avx_01, vect_double_reduce_01,
vect_double_reduce_special_values_01) while the compiler was otherwise correct
without -Oodeadstore. The bug is pre-existing and independent of -OoPURE.

Fix
---
Mirror the store-side workaround for the read side: copy the incoming s into a
memory-backed scalar temp with a PLAIN assignment first, and seed the packed
accumulator from that temp:

    seedtemp := s;                 { DFA-visible read of s keeps its def live }
    acc := reduce_init( acctemp, seedtemp )

The plain  seedtemp := s  assignment is an ordinary loadn read of s that DFA
tracks, so the incoming def of s stays live and DSE no longer removes it. Emitted
code is otherwise identical (one extra scalar copy that the register allocator
folds away in the common case).

Test
----
New runtime fixture unleashed/tests/testfiles/optvect/vect_reduce_deadstore_01.pp
forces -OoVECTORIZE -Oodeadstore via %OPT and checks single/double sum and dot
reductions against a strict sequential (downto) oracle over exactly-representable
inputs for every trip count; it fails before this change and passes after.

Full unleashed suite bit-clean both normally and with -Oodeadstore -OoVECTORIZE
forced suite-wide (1011 pass / 5 pre-existing known fails, no new failures); all
30 optvect tests green with the flags forced; all 12 *_check.sh codegen scripts
PASS.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…atic/global accumulators

Lands the three remaining -OoFINALVALUE follow-ups, soundness-first (a wrong
final value or wrongly-deleted loop is a miscompile, so the pass still leaves
any non-provable loop alone).

* 64-BIT COUNTERS: the counter may now be any integer up to 64 bits (max size
  raised 4 -> 8). The exact trip count is not needed: b-a+1 is computed in
  64-bit two's-complement (the true iteration count reduced mod 2^64) and each
  accumulator product is truncated to the accumulator's own width w<=64, so
  (count*c) mod 2^w is bit-identical to the repeated addition; a pointer
  stride's count*stride element count is likewise correct mod 2^64.

* STATIC / GLOBAL accumulators: a unit-level (static) variable may be an
  accumulator (never the counter -- a static counter's cross-unit uses the
  whole-routine deadness scan cannot see). Sound because the matched body has
  no calls and the routine no exception paths, so nothing observes the global's
  intermediate states cross-routine within this thread; the closed form stores
  the identical final value. different_scope (normally set on any global
  reached from a subroutine) is no longer treated as the nested-capture hazard
  -- that check now applies to locals only; addr_taken / volatile still reject,
  and threadvars / externals stay out.

* RANGE CHECKING (-Cr, without -Co): re-enabled, restricted to native
  full-range integer accumulators (int64/qword) whose s:=s+c performs no
  narrowing and so is never range-checked, matching the nf_internal closed form
  bit-for-bit. Sub-native / subrange / pointer accumulators are range-checked
  on their narrowing store, so they are declined (loop kept). Bound range
  faults are preserved (the guard copies the original bound expressions
  verbatim), and the empty-dead-loop shortcut is disabled under -Cr so a
  narrowing bound still faults at loop entry.

* -Co (overflow checking) stays DISABLED by design: a native accumulator loop
  traps on the overflowing iteration while the wrap-around closed form does
  not, and reproducing that per-iteration trap from a single closed-form step
  is not robustly sound (sub-native trap-freedom relies on a fragile promotion
  detail; a native accumulator's count*c can overflow int64 in exactly the
  trapping case). The pass exits early whenever overflow checking is on.

Tests: new runtime fixtures fv_counter_64bit_01, fv_global_accum_01,
fv_rangecheck_native_01; finalvalue_check.sh extended with codegen asserts
(64-bit-counter / native / global loops deleted, deleted under -Cr;
sub-native kept under -Cr; every loop kept under -Co). fv_no_transform_01
updated (global accumulator is now transformed). Full unleashed suite 1013
pass / 0 fail, byte-identical normally and with -OoFINALVALUE forced suite-wide.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extend TX86AsmOptimizer.OptSibCall to recognise result forwarding through
more than one location. The old matcher accepted only a single callee-saved
register identity pair (rax->cs->rax). It now walks a chain of forward-saves
that stage one or more integer return registers (rax, or rax:rdx for a
128-bit result) through callee-saved registers and/or rsp-relative frame
slots, each with a matching restore before the teardown. Sources and
destinations must be pairwise distinct and every save must be restored before
any teardown, so the save/restore round-trip is a proven identity on the
return registers; the transform then hoists the teardown, turns the call into
a jmp, and drops the (now dead) forwarding moves exactly as before. This is
the shape the register allocator emits for a 128-bit rax:rdx record result
staged through two frame slots, and for a result split across a callee-saved
reg and a stack slot.

Also harden a pre-existing soundness hole in the frame-teardown gate
(CurrentProcAllowsSiblingTailFrameReuse): symtable_has_addrtaken only tracks
named locals/parameters, so compiler temporaries whose address escapes to the
callee -- an open array of const, a @temp passed by reference -- slipped
through and the frame was torn down before the callee read them (dangling
stack). Scan the routine and decline when any lea/mov materialises a
frame-relative address into a general register. This fixes a real miscompile
that surfaced only under a forced suite-wide -O2 -OoSIBCALL
(match_as_expression_in_arg_01); it does not affect the eligible register-only
mutual-recursion / continuation cases.

Tests: sibcall_forward128_01.pp (deep 128-bit + single-reg forwarding proving
O(1) stack and correct results), sibcall_frameaddr_negative_01.pp (escape
cases decline and stay correct), and new firing/decline codegen assertions in
sibcall_check.sh (multi-location forward jmps, non-tail and frame-address
declines).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…icter case-exhaustiveness check in optloop.pas

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- ProducedExePath appended .exe unconditionally, but unix binaries have no
  extension: every run-phase test reported 'compiled but exe not found'
- inject -Facthreads on non-Windows targets so the threaded-feature tests
  (parallel for, async/await, lock, thread static) get the cthreads driver
  the compiler itself only hints about

With these, 1200/1202 tests pass on x86_64-linux; the two failures
(inline_vars widestring RTTI name, asyncawait reraise flake) reproduce
identically on the pre-merge compiler.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@fibodevy fibodevy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Adds regression tests under unleashed/tests/testfiles/dead_store_normalize/ (all fail on the old compiler)"

All four pass on current main (-O3 -OoDEADSTORE):

[1/4] dead_store_normalize\dsnorm_ifexpr_nested_loops_01.pp ... PASS
[2/4] dead_store_normalize\dsnorm_match_in_loop_01.pp ... PASS
[3/4] dead_store_normalize\dsnorm_ifexpr_in_loop_01.pp ... PASS
[4/4] dead_store_normalize\dsnorm_funcref_compiles_01.pp ... PASS

The IE 200108231 fix landed upstream about a month ago (!1442), so the normalize() bug has been gone there since then. Today's sync pulled it into our main (acc89a1), so as of today it's fixed on our side too - which is why these pass instead of fail. "all fail on the old compiler" only held against the pre-sync base.

So the fix part is redundant now - unless I've overlooked something, in which case can you point to the case it still handles that acc89a1 doesn't? The regression tests themselves are welcome (they pass and lock the fix in), so I'd just take those.

Also worth checking the other fixes in the PR the same way - since this one was already resolved upstream, some of the others may have landed there too and be redundant after today's sync.

@fibodevy fibodevy self-assigned this Jul 9, 2026
@fibodevy fibodevy added the enhancement New feature or request label Jul 9, 2026
@fibodevy fibodevy requested review from fibodevy and removed request for fibodevy July 9, 2026 20:17
@joaopauloschuler

joaopauloschuler commented Jul 9, 2026

Copy link
Copy Markdown
Author

Many thanks for the feedback. I will work on it (eventually).

I will review: the nadd.pas const-fold spurious overflow, the -Oodeadstore × -OoVECTORIZE stale-seed miscompile, the testtool unix run-phase fix, and the case-exhaustiveness fixup.

joaopauloschuler and others added 4 commits July 10, 2026 18:00
Add overloaded declarations bound to the same external symbols so
callers can pass a variable directly instead of a pointer for the
out-parameters of clGetPlatformInfo, clEnqueueMapBuffer,
clEnqueueMapImage, clGetDeviceInfo, clCreateContext,
clCreateContextFromType, clGetContextInfo, clGetCommandQueueInfo,
clSetCommandQueueProperty, clCreateBuffer, clCreateImage2D,
clCreateImage3D, clGetSupportedImageFormats, clGetMemObjectInfo,
clGetImageInfo, clCreateSampler, clGetSamplerInfo,
clCreateProgramWithSource, clCreateProgramWithBinary,
clGetProgramInfo, clGetProgramBuildInfo, clCreateKernel,
clCreateKernelsInProgram, clGetKernelInfo, clGetEventInfo and
clGetEventProfilingInfo.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add a clCreateCommandQueue overload taking 'var errcode_ret: cl_int'
alongside the existing pcl_int version; mark both with 'overload'.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…arus checkout

rebuildu.sh chains the full toolchain rebuild (compiler clean cycle
seeded via fpc -PB, RTL, packages, and the LCL when ../lazarus exists),
detects the host CPU instead of assuming x86_64, and warns about stray
package PPUs shadowing rtl/units.

lazbuildu.sh now passes --lazarusdir pointing at the sibling ../lazarus
checkout (LAZDIR overridable) so dependency packages are rebuilt from
source with the in-tree compiler instead of failing with "PPU Invalid
Version" against the system Lazarus install's precompiled units.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fpmake's NeedsCompile never compares package PPUs against the compiler
or RTL (it only recompiles when a dependency package was built in the
same fpmake session, or the unit's own sources/includes are newer), so
a default rebuild left packages compiled by the previous compiler
against the previous RTL next to the fresh toolchain. Make the
packages clean unconditional; --no-clean-packages restores the old
fast path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@fibodevy fibodevy force-pushed the main branch 2 times, most recently from 5178d48 to 9e03840 Compare July 14, 2026 01:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants